8 A Relation between Complexity and

We derive two asymptotic formulas relating the Kolmogorov complexity of strings over a nite alphabet to the entropy of a discrete Markov information source that generates the strings. In 2], Beyer, Stein, and Ulam discussed several notions of complexity of integers and made a conjecture relating Kolmogorov complexity and information theoretic entropy. We will state this conjecture, after summarizing the basic notions that it refers to. More comprehensive introductions are 3] and 6] for Kolmogorov complexity, and 1] for information theory. Let A be an algorithm, i.e. a Turing machine, that transforms binary strings into binary strings. (The restriction to binary strings is for simplicity ; all the deenitions and results given here easily extend to strings over any nite alphabet.) The complexity of a string x relative to A, K A (x), is the length of the shortest string w such that A(w) = x, or if no such string exists, it is 1. Similarly, if A is an algorithm that transforms pairs of binary strings into binary strings, the conditional complexity of x with respect to y, K A (xjy), is the length of the shortest string w such that A(w; y) = x, or if no such string exists, it is 1. As in 2], we will assume A is of this form, and we will consider conditional complexities K A (xjn), where n = jxj, the length of x. Let S be a discrete 0-memory binary information source with probability(0) = p and probability(1) = 1 p. That is, S generates a sequence of Bernoulli trials whose outcomes are 0 or 1, and for a binary string x of length n with m 0's, the probability of x, pr(x), is p m (1 p) nm. The entropy H of S is p log p (1 p) log(1 p). (All our logarithms are base 2.)